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I ABSTRACT: We prove in this paper that every endomorphism of the map- 

' ping class group of certain orientable surfaces onto a subgroup of finite index 

fSJ , is in fact an automorphism. 

^ . 1. Introduction 

■ Let 5 be a compact connected orientable surface. The mapping class 

(-^ , group M.S of the surface 5 is the group of isotopy classes of orientation 

preserving diffeomorphisms 5 — > S*. The extended mapping class group Aig 
of S is the group of isotopy classes of all diffeomorphisms — > S". Note that 
the isotopy classes of orientation reversing diffeomorphisms are also included 
in A4*g, and hence A4s is a subgroup of A4*g of index two. 

Recall that a group G is called residually finite if for each x ^ 1 inG there 
1/^ I exists a homomorphism / from G onto some finite group such that f{x) is 

CN ■ nontrivial. Equivalently, there is some finite index normal subgroup of G 

, that does not contain x. G is called hopfian if every surjective endomorphism 

I of G is an automorphism. It is well known that finitely generated residu' 



On ■ ally finite groups are hopfian [LS|. G is called cohopfian if every injective 

I endomorphism of G is an automorphism. 

■ The mapping class group of an orientable surface is finitely generated 

, [0) @ and residually finite ||^, |ll|. Hence it is hopfian. N.V. Ivanov and 

J.D. McCarthy ||IM| proved that A4s is also cohopfian. Author and J.D. 
McCarthy proved that if (p : Ms ^ Ms is a homomorphism such 

that (j){Ms) is a normal subgroup and Ms/4>iMs) is abelian, then (p is an 
^ ' automorphism. 



> 

X 

^ . In this paper, we prove further that if (p is an endomorphism of the map- 

ping class group Ms onto a finite index subgroup, then <j) is in fact an 
automorphism, with a few exceptions. The proof of this result relies on a 
result of R. Hirshon, which states that if (j) is an endomorphism of a finitely 
generated residually finite group G such that 0(G) is of finite index in G, 
then (j) restricted to (j)^{G) is an injection for some n. 

D.T. Wise | [W[ | gave an example of a finitely generated residually finite 
group G and an endomorphism ^ of G such that the restriction of $ to 
$"(G) is not injective for any n, answering a question of R. Hirshon in 
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negative. It might be interesting to consider the same question for mapping 
class groups of surfaces. 

2. Endomorphisms of mapping class groups 

Let S be a compact connected oriented surface of genus g with h bound- 
ary components. For any simple closed curve a on 5, there is a well known 
diffeomorphism, called a right Dehn twist, supported in a regular neighbor- 
hood of o. We denote by ta the isotopy class of a right Dehn twist about 
o, also called a Dehn twist. Note that ftaf~^ = if {a) foi' orientation 
preserving mapping class /. 

The pure mapping class group VM.s is the subgroup of A^^ consisting of 
those orientation preserving mapping classes which preserve each boundary 
component. 

For a group G and a subgroup of it, we denote by Cg{H) the centralizer 
of H in G. The center of G is denoted by C{G). 

Theorem 1. Let G be a finitely generated residually finite group, and let (j) 
be an endomorphism of G onto a finite index subgroup. Then there exists 
an n such that the restriction of (j) to (j)^{G) is an injection. 

Theorem 2. Let S be a compact connected orientable surface of genus g 
with b boundary components. Suppose, in addition, that if g = then 6 > 5, 
if g = 1 then b>3, and if g = 2 then 6 > 1. Then any isomorphism between 
two finite index subgroups of the extended mapping class group is the 
restriction of an inner automorphism of A4*g. 

Theorem |l] was proved by R. Hirshon ( [Q ) , and Theorem ^ was proved 
by N.V. Ivanov [Q for surfaces of genus at least two and by the author 
for the remaining cases. Since the mapping class group Ms is normal in 
Mg, we deduce the following theorem. 

Theorem 3. Let S be a compact connected orientable surface of genus g 
with b boundary components. Suppose, in addition, that if g = then b > 5, 
if g = 1 then b>3, and if g = 2 then 6 > 1. Then any isomorphism between 
two finite index subgroups of the mapping class group A4s is the restriction 
of an automorphism of A4s- 

Lemma 4. Let S be a closed orientable surface of genus two and let T be a 
finite index subgroup of Ms- Then the center G(T) ofV is equal to Fn (cr), 
where a is the hyperelliptic involution. 

Proof: Since the subgroup (cr) = {1,0-} is the center of Ms-, its intersec- 
tion with F is contained in the center of F. 

Now let / G C(F) and let be the index of F in Ms- Since G F for 
all simple closed curves a, we have t^(^a) ~ f^af~^ — '^a ■ follows that 
/(a) = a (cf. |IM|] ) . Hence, ftaf~^ = if{a) = ta- Since Ms is generated by 
Dehn twists, / G C{Ms) = {cr)- □ 
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We are now ready to state and prove the main result of this paper. 

Theorem 5. Let S be a compact connected orientable surface of genus g 
with b boundary components. Suppose, in addition, that if g = then b ^ 
2,3,4, and if g = 1 then b ^ 2. If (p is an endomorphism of Ads such that 
(j){M.s) is of finite index in TWs, then (j) is an automorphism. 

Proof: If S" is a (closed) sphere or a disk, then A^s is trivial. Clearly, the 
conclusion of the theorem holds. 

Suppose first that S is a torus with 6 < 1 boundary component. It is 
well known that Ms is isomorphic to S'L2(Z). The commutator subgroup 
of (Z) is a nonabelian free group of rank 2 and its index in (Z) is 12. 
Let us denote it by F2. 4'{F2) is contained in F2 as a finite index subgroup. 
If this index is k, 4'{F2) is a free group of rank k + 1. Since there is no 
homomorphism from F2 onto a free group of rank > 3, it follows that k = 1. 
That is, 4'{F2) = F2. In particular, (/)(S'L2(Z)) contains F2. The fact that (j) 
is an automorphism in this case was proved in ||KM |. 



Suppose now that S is not one of the surface above and not a closed a 
surface of genus 2. Let us orient S arbitrarily. Since Al^ is finitely gener- 
ated and residually finite, there exists an n such that the restriction of (j) 
to (p'^i^Ais) is an isomorphism onto (l)'"~^^{A4s)- Note that the subgroups 
(j)'^{A4s) ^-nd (p^^^^Ms) are of finite index in Ms- Hence, there is an au- 
tomorphisms a of Ms such that the restrictions of a and (j) to (^"'{Ms) 
coincide. 

Let be the index of (j)^^{Ms) in ^^5. For any simple closed curve a on 
5, t^ is contained in (l)'^{Ms)- Hence, a{t^) = 4>{ta)- Let / € Ms be any 
element. Then 

= *a-iW/))(a)- 



Hence, ((/>(/)) (a) = /(a) for all a (cf. @). It follows that /-ia-i(0(/)) 
commutes with all Dehn twists. Since VMs is generated by Dehn twists, 
it is in Cxg(PAl5), the centralizer of VMs in A^^. But (^^^^(PAIs) is 



trivial [IM]. Hence, a~^(0(/)) = /. Therefore, (p = a. In particular, (p is an 
automorphism. 

Suppose finally that 5 is a closed surface of genus two. Let i? be a sphere 
with six holes. Then Mr is isomorphic to the quotient of Ms with its center 
(d), where a is the hyperelliptic involution (cf. |BH|| ) . Let us identify Mr 
and Ms/{cr), and let vr : A^^ — > Mr be the quotient map. Since (p{cr) is in 
the center of (l){Ms), either 4>{(j) = a ov (p{a) = 1 by the lemma above. 
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If 0(cr) = o", then (j) induces an endomorphism <I> of A4r, such that ncp = 
$7r. Then we have a diagram in which all squares are commutative: 



1 — 




- Ms 


^ Mr - 


1 












1 — 




Ms 


^ Mr - 


1 



where I is the identity homomorphism. Since the image ^{Mr) of ^ is of 
finite index, $ is an automorphism by the first part. By 5-lemma, (p is an 
automorphism. 

If (j){cr) = 1, then (j) induces a homomorphism (p : Mr — > Ms such that 
(f)7r = (p. The image of the endomorphism $ = iTcp oi Mr has finite index. 
Since i? is a sphere with six holes, $ is an automorphism by the first part. 
Then, 4> is an automorphism, and hence a = 1. This contradiction finishes 
the proof of our theorem. □ 

Remark: If 5" is a sphere with two holes, then A4s is a group of order two, 
and if is a sphere with three holes, then TW^ is isomorphic to the symmetric 
group on three letters. Hence, in these cases the trivial homomorphism is 
an endomorphism onto a finite index subgroup which is not automorphism. 
We do not know if the conclusion of Theorem ^ holds if a sphere with four 
holes and a torus with two holes. 
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